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This  paper  presents  a  transferable  and  adaptable  approach  for  selecting  a  suitable  Multi-Criteria  Analysis 
(MCA)  method  covering  the  stages  of  the  ‘MCA  cycle’  for  decision  making  in  the  renewable  energy  sector. 
Selecting  a  suitable  MCA  method  for  an  individual  project  or  case  study  is  a  challenge  due  to  the 
complexity  of  participatory  sustainable  decision  making  in  this  sector,  as  well  as  due  to  the  complexity 
and  large  number  of  existing  MCA  methods.  The  methodology  presented  in  this  paper  consists  of  both, 
generic  and  case-study  specific  parts. 

The  paper  concludes  that  the  methodology  provides  a  transferable  approach  to  systematically  select  a 
MCA  method  and  to  justify  this  selection.  The  methodology  supports  focusing  on  specific  aspects,  which 
are  considered  important  and  relevant  for  decision  making  processes.  Furthermore,  the  case  study's 
generic  decision  making  situation  allows  the  methodology  to  be  applied  broadly  within  the  renewable 
energy  sector.  Moreover,  the  MCA  methods  and  selection  criteria  identified  and  reviewed  provide  a  broad 
application  potential  to  assist  MCA  method  selection  in  the  energy  and  renewable  energy  sectors. 
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1.  Introduction 

Multi-criteria  analysis  (MCA)  can  be  defined  as  formal  or  struc¬ 
tured  approaches  for  individuals  or  groups  to  determine  overall 
preferences  among  alternative  options  by  taking  account  of  multiple 
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criteria  and  indicators  (C&I)  [1-3],  MCA  approaches  show  explicitly 
how  the  alternative  options  contribute  to  the  various  C&I  and  contain 
a  judgment  process,  which  usually  comprises  a  relative  weighting 
process.  MCA  approaches  can  be  exercised  for  identifying  a  single 
most  preferred  option  or  for  a  ranking  of  alternative  options,  a  short 
list  with  a  limited  number  of  options  for  further  assessment,  as  well  as 
for  determining  acceptable  and  unacceptable  options  [2].  They  have 
become  increasingly  popular  in  decision  making  on  sustainable 
developments  and  on  energy  systems  due  to  their  ability  to  consider 
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and  simultaneously  evaluate  a  number  of  economic,  environmental, 
social  and  technological  aspects,  which  complex  systems  require  [4], 
For  that  purpose  MCA  approaches  offer  algorithms  to  systematically 
aggregate  multi-dimensional  information  to  reduce  complexities  in  a 
transparent  way  (e.g.  [5]).  They  proved  to  be  of  assistance  in 
comprehensive  decision  processes  and  in  overcoming  implementation 
barriers  by  structuring  the  problem,  identifying  less  robust  and 
uncertain  components  of  energy  systems  and  involving  different 
stakeholders  in  challenging  and  difficult  decision  making  situations 
[6], 

In  literature,  there  are  several  approaches  to  classify  MCA 
approaches.  A  popular  form  of  MCA  is  Multi-Criteria  Decision 
Analysis  (MCDA),  which  is  also  known  as  multi  attribute  decision 
analysis  (MADA)  [2],  Further  notable  is  that  the  terms  ‘MCDA’,  as 
well  as  ‘MCDM’  (Multi-Criteria  Decision  Making)  are  often  syno¬ 
nymously  used  for  MCA. 

Moreover,  Buchholz  et  al.  [6]  and  others  [7-9]  are  describing 
two  main  categories:  Multi  Objective  Decision  Making  (MODM) 
and  Multi  Attribute  Decision  Making  (MADM)  approaches.  The 
main  difference  between  the  two  groups  of  approaches  is  the 
number  of  possible  alternatives,  which  are  evaluated.  For  MODM 
an  indefinite  number  of  alternatives  are  defined  by  a  set  of 
constraints  on  a  vector  of  decision  variables  resulting  in  an 
optimized  alternative.  On  the  other  hand,  for  MADM  approaches, 
which  are  usually  employed  for  environmental  decision  making 
problems  [10],  a  certain  finite  number  of  alternatives,  which  are 
given  explicitly,  are  developed  and  assessed  against  a  set  of 
principles  or  C&I.  These  are  often  expressed  as  qualitative  and/or 
quantitative  attributes  [1,7,10,11], 

According  to  Belton  and  Stewart  [1]  (adopted  by  others,  e.g. 
[6,7])  MADM  approaches  can  generally  be  differentiated  into  three 
broad  categories  to  illustrate  their  application  ranges  more 
directly.  These  three  categories  are:  (i)  value  measurement  mod¬ 
els,  (ii)  goal,  aspiration  and  reference-level  models  and  (iii)  out¬ 
ranking  models.  The  approaches  in  the  first  category  follow  the 
Multi  Attribute  Value  Theory  (MAVT)  and  its  extension  the  Multi 
Attribute  Utility  Theory  (MAUT),  which  works  with  utility  func¬ 
tions  for  additional  inclusion  of  risks  and  uncertainties.  Belton  and 
Stewart  [1  ]  describe  goal,  aspiration  and  reference-level  models  as 
goal  programming  methods,  which  employ  algorithms  based  on 
mathematical  programming  to  meet  the  goals,  aspirations  or 
reference  levels  as  much  as  possible.  Outranking  models  compare 
scenarios  and  alternatives  against  each  criterion  and  its  indicator 
on  a  pairwise  basis  [12],  In  this  process  incomparabilities  are 
identified,  as  well  as  preferences  and  indifferences  are  assessed. 
The  aggregation  of  preferences  in  respect  to  each  criterion  and  its 
indicator  results  in  a  scenario  or  alternative,  which  outranks 
others.  However,  in  contrast  to  MAUT  the  aim  is  usually  not  to 
identify  a  single  highest  scoring  alternative,  but  rather  to  involve 
multiple  stakeholders  in  a  process  to  advise  each  other  [13], 

MCA  methods  can  be  considered  for  different  stages  of  multi¬ 
criteria  decision  making.  Wang  et  al.  [14]  reviewed  MCDA  methods 
for  C&I  selection,  C&I  weighting,  evaluation,  and  final  aggregation. 
However,  with  respect  to  the  latter  stage  Guitouni  and  Martel  [15] 
believe  that  aggregation  procedures  and  MCA  methods  are  con¬ 
fusing  concepts  and  have  been  used  in  the  same  way.  This  is  not 
surprising  since  some  methods,  such  as  Analytic  Hierarchy  Process 
(AHP)  can  contribute  to  more  than  one  decision  making  stage. 
Moreover,  Buchholz  et  al.  [6]  illustrate  the  stages  of  a  ‘MCA  cycle’, 
which  is  a  separate  cycle  within  an  ‘adaptive  multi-stakeholder 
management  cycle’.  These  stages  commonly,  but  not  exclusively 
comprise:  (i)  C&I  weighting  or  ranking,  (ii)  assessment  and 
ranking  of  alternatives  and  (iii)  sensitivity  analysis. 

Against  this  background,  this  paper  presents  a  methodology  to 
select  suitable  MCA  methods  covering  the  stages  of  the  ‘MCA  cycle’ 
for  decision  making  in  the  renewable  energy  sector,  which  is 


challenging  due  to  the  mentioned  complexity  of  participatory 
sustainable  decision  making  in  the  energy  sector,  as  well  as  due 
to  the  large  number  and  complexity  of  MCA  methods. 

2.  Methodology  for  MCA  method  selection 

The  methodology  presented  in  this  paper  builds  on  literature 
(e.g.  [7,16]).  For  instance,  Guitouni  and  Martel  [15]  provide  a 
framework  to  develop  a  selection  methodology  and  tentative 
guidelines  to  support  selecting  an  appropriate  MCDA  method. 
Furthermore,  Buchholz  et  al.  [6]  analyzed  the  applicability  of  four 
MCA  tools  for  bioenergy  systems  assessments  by  scrutinizing  their 
structural  approach  using  a  set  of  nine  criteria  to  evaluate 
methodological  implications,  technical  aspects  and  the  application 
value  of  MCA  methods. 

Following  the  methodology  recommendations  from  the 
reviewed  literature,  the  methodology  presented  in  this  paper 
consists  of  both,  generic  and  case-study  specific  parts.  First,  MCA 
methods  and  criteria  to  select  MCA  methods  from  literature  were 
identified  and  reviewed.  Then,  the  methodology  was  applied  in  a 
case  study  involving  the  main  steps:  development  of  a  generic 
decision  making  situation,  development  of  a  set  of  appropriate 
criteria  for  MCA  method  selection,  pre-selection  of  MCA  methods 
and  scoring  of  the  pre-selected  MCA  methods  against  the  devel¬ 
oped  selection  criteria.  In  the  subsequent  sections  these  processes 
are  described  and  discussed  in  detail. 

2.1.  Identification  of  weighting  and  MCA  methods  applied  in  the 
energy  and  renewable  energy  sectors 

MCA  methods  have  been  extensively  addressed  in  academic 
articles  covering  the  energy  sector  in  general,  as  well  as  the 
renewable  energy  sector.  Table  1  summarizes  reviewed  literature 
on  weighting  and  MCA  approaches  applied  in  these  sectors.  The 
table  illustrates  only  a  certain  fraction  of  studies  in  these  sectors. 
However,  in  contrast  to  similar  summaries  (e.g.  [4])  it  focuses  on 
more  recently  applied  methods,  as  well  as  stakeholder  involve¬ 
ment.  For  a  detailed  review  of  methods  in  the  different  stages  of 
multi-criteria  decision  making  for  sustainable  energy  we  recom¬ 
mend  Wang  et  al.  [14],  Further  rather  theoretical  studies  in  this 
field  include  for  instance  Chatzimouratidis  and  Pilavachi  [17], 
Pilavachi  et  al.  [18],  Chatzimouratidis  and  Pilavachi  [19], 

It  can  be  concluded  that  the  vast  number  of  applied  MCA 
methods  to  support  sustainable  decision  making  in  the  energy 
sector  in  general,  as  well  as  the  renewable  energy  sector  makes  it 
imperative  to  take  a  systematic  approach  to  select  MCA  methods 
for  individual  projects  or  case  studies.  Further  notable  is  that 
frequently  several  different  suitable  weighting  and/or  MCA  meth¬ 
ods  are  applied  to  a  decision  making  problem  to  look  at  problems 
from  various  angles.  This  can  help  cross-checking  and  verifying  the 
validity  of  MCA  method  results  before  arriving  at  the  final  ranking 
of  alternatives. 

2.2.  Criteria  for  selecting  MCA  methods 

Criteria  for  selecting  MCA  methods  from  literature  against 
which  MCA  methods  can  be  scored  are  described  and  summarized 
(Table  2)  in  this  section. 

Guitouni  and  Martel  [15]  provide  general  tentative  guidelines 
to  help  choosing  an  appropriate  MCA  (specifically  MCDA)  method. 
Their  first  guideline  is  to  determine,  whether  a  decision  is  made  by 
one  or  many  decision  makers.  Second,  the  preference  elucidation 
and  modeling  has  to  be  considered.  This  guideline  is  divided  into: 

(a)  the  preference  elucidation  mode  (e.g.  pairwise  comparison), 

(b)  the  timing  of  preference  elucidation,  (c)  the  global  decision 
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Table  1 

Summary  of  weighting  and  MCA  methods  applied  in  the  energy  and  renewable  sectors. 


Scope 

Weighting  and/or  MCA  method 

Special  features 

Stakeholder  involvement 

Source 

Sustainability 
framework  to 
assess  bioenergy 
systems 

Not  specified  MCDA  method 

Least  cost  optimization  models  for  scenarios. 
Life  cycle  impact  assessment  (LCIA)  for  C&I 
performance  assessments.  Numeral  Unit  Spread 
Assessment  Pedigree  (NUSAP)  approach  to 
classify  qualitative  and  quantitative 
uncertainties. 

Decision  conferencing  for  C&l  selection 
(different  stakeholders).  Experts  for  C&I 
performance  assessments 

[20] 

Sustainability 
assessment  to 
select  a  country's 
energy  system 

MCA  based  on  Analysis  and  Synthesis 
of  Index  at  Information  Deficiency 
(ASPID)  methodology  and  a  General 
Sustainability  Index 

Comparison  of  MCA  to  single  criterion  analysis. 
Stochastic  modeling  of  uncertainty,  as  well  as 
normalization  of  indexes  by  using  Non-numeric 
(ordinal),  Non-exact  (interval)  and  Non- 
complete  information  (NNN-information)  with 
respective  reliability  and  probability 

No 

[8] 

Renewable  energy 

S1MOS  and  Preference  ranking 

Resource-intensive  combination  of  scenarios 

Scenario  development  and  improvement  of 

[4] 

scenarios  and 

organization  method  for  enrichment 

and  PMCA.  Narrative  storylines  for  scenarios. 

developed  C&I  (energy  stakeholders  and 

participatory 
multi-criteria 
analysis  (PMCA) 

evaluation  (PROMETHEE)  II 

C&I  performance  assessments  based  on  own 
modeling,  existing  databases  and  expert 
assessments.  ‘Silent  negotiation’  technique  for 
C&I  ranking 

experts).  Experts  and  stakeholders  for  C&I 
performance  assessments  and  weighting. 

MCA  for  bioenergy 

Analytic  Hierarchy  Process  (AHP), 

Criteria  for  MCA  method  selection.  Sensitivity 

Stakeholders  and  experts  involved  in  C&I 

[6] 

systems 

DELTA  ,  PROMETHEE  II,  Novel 
Approach  to  imprecise  Assessment 
and  Decision  Environments  (NAIADE) 

analyses  on  C&I  weighting  and  performance 
assessment  results 

weighting  and  performance  assessments 

Appraisal  of 
national  energy 
scenarios  - 
renewable  energy 

SIMOS  and  PROMETHEE  (Decision  Lab) 

Stakeholder  involvement  in  scenario  and  C&I 
development,  individual  C&I  weighting  and 
discussion  of  MCA  results.  Experts  involved  in 
C&I  performance  assessments 

[21] 

Sustainable 
development  of  a 
city's  energy 
system 

MCA  based  on  fuzzy  sets  synthesis 
technique 

No 

[22] 

Alternative  power 
generation 
scenarios 

PROMETHEE 

Comparison  of  MCDA  and  Cost-Benefit-Analysis 
(CBA) 

Official  authorities  involved  in  scenario 
development 

[23] 

Small-scale  energy 

MACBETH  (Measuring  Attractiveness 

Performance  assessments  involving  LCA  and 

Households,  energy  professionals  and  local 

[9] 

technology 

by  a  Categorical  Based  Evaluation 

CBA  (including  calculation  of  Net  Present 

council  members  involved  in  C&I  performance 

applications  in 
local 

governments 

TecHnique) 

Values) 

assessments  and  weighting 

Optimization  of 

Elimination  et  choice  translating 

Sensitivity  analysis  in  relation  to  the  preference, 

Local  authorities  and  private  actors  involved  in 

[24] 

decentralized 
energy  systems 

reality  (ELECTRE)  III 

indifference,  veto  thresholds  and  importance 
coefficients 

C&I  weighting  and  sensitivity  analysis 

Formulation  of 

MCDA  with  linguistic  variables  use 

Linguistic  decision  support  system  (L-DSS) 

Working  groups  with  national  energy  actors 

[25] 

sustainable 
technological 
energy  priorities 

(fuzzy  set  theory) 

incorporating  linguistic  ordered  weighted 
averaging  (LOWA),  the  2-tuple  LOWA  and  the 
OWMAX  (ordered  weighted  minimum 
operators)  to  deal  with  imprecision 

involved  in  C&I  and  alternatives  selection,  C&I 
weighting  and  performance  assessments 

Public  and 
stakeholder 
perceptions  of 
bioenergy 
scenarios 

Simple  additive  approach 

Policy  stakeholders  and  informed  local 
members  of  the  public  involved  in  scenario 
development,  C&I  weighting  and  performance 
assessments,  as  well  as  appraisal  of  final 
weighted  scores 

[26] 

Energy  policy  and 
production  site 
planning  for  a  city 

Integrated  fuzzy  VIKOR 
(VIsekriterijumsko  KOmpromisno 
Rangiranje)  and  AHP  methodolgy 

Integrated  methodology  to  select  energy 
alternatives  and  sites 

Experts  involved  in  C&I  weighting 

[27] 

Land  suitability 
assessment  for 
spatially 
distributed 
anaerobic 
digester 

AHP 

Geographic  Information  System  (GIS)  modeling 
to  locate  optimal  sites.  Performance 
assessments  for  distance-related  C&I  combined 
with  weights  obtained  from  AHP  application 

No 

[28] 

Evaluation  of 
alternative  fuels 
for  electricity 
generation 

Analytic  Network  Process  (ANP) 

(Super  Decisions) 

LCA  use 

Experts  involved  in  C&I  weighting 

[29] 

making  preference  structure  obtained  using  the  aggregation 
procedure  and  resulting  from  that  (d)  the  specific  alternative 
order  type.  Thirdly,  the  decision  makers'  desired  outcomes  have 
to  be  taken  into  account.  An  example  for  such  an  outcome  can  be  a 
ranking  of  alternatives.  The  fourth  guideline  is  about  a  MCA 
method's  ability  to  process  available  input  data  properly.  Required 


data  should  easily  be  provided  by  stakeholders.  Where  qualitative 
data  is  used,  it  can  make  a  significant  difference  to  the  quality  of 
decision  making,  if  a  MCA  method  is  capable  to  incorporate  that 
data.  Another  guideline  is  about  dealing  with  the  compensation 
degree  of  a  MCA  method.  However,  Guitouni  and  Martel  [15] 
admit  that  this  aspect  is  difficult  to  evaluate.  They  also  advise  to 
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Table  2 

Criteria  for  MCA  method  selection  from  literature. 


Selection  criteria 

Source 

Multi-stakeholder  inclusion 

[6,7,15] 

Determining  of  C&I  weights 

[7,15,16] 

Handling  of  qualitative  and  quantitative  data 

[6,7,15,16] 

Shape  and  form  of  final  results 

[15,16] 

Compensation  degree 

[15] 

Meeting  the  method's  fundamental  hypothesis 

[15] 

Support  by  a  software  package 

[15] 

Measures  to  deal  with  uncertainty 

[6,7] 

Country  of  application 

[7] 

Type  of  data  used 

[7] 

Nature  and  context  of  the  problem 

[7] 

Spatial  scale 

[7] 

Type  and  selection  of  stakeholders 

[7,13] 

Inclusion  of  C&I  hierarchies 

[6] 

Use  of  thresholds 

[6] 

Ease  of  computation 

[6,13] 

Dynamic  re-evaluation 

[6] 

Transparency  and  likely  user  acceptance  of  results 

[6,13] 

Communication  of  decision  process  and  results 

[6,13] 

Core  process 

[16] 

Necessity  to  quantify  the  relative  importance  of  C&I 

[16] 

Number  and  type  of  outranking  relations 

[16] 

Control  of  consistency 

[16] 

bear  in  mind  how  MCA  methods  meet  their  fundamental  hypoth¬ 
eses.  However,  the  information  this  criterion  provides  can  be 
regarded  as  being  more  descriptive  and  less  suitable  for  MCA 
method  comparison  and  selection.  The  final  guideline  they  men¬ 
tion  is  about  the  availability  of  software  packages  to  apply  a  MCA 
method.  The  authors  raise  concerns  that  in  many  cases  decision 
aid  package  software  is  applied  without  deeper  knowledge  of  the 
procedure.  On  the  other  hand,  they  argue  that  it  can  occur  that  a 
MCA  method,  which  would  be  suitable  for  a  decision  making 
situation  is  not  employed  simply  because  a  software  package  is  not 
user-friendly  and/or  does  not  allow  integration  of  a  variety  of  MCA 
methods. 

Mendoza  and  Martins  [7]  critically  reviewed  and  systematized 
applied  MCA  (specifically  MCDM)  methods  in  natural  resource 
management  using  a  number  of  criteria  and  characteristics.  The 
most  relevant  of  those  indicate,  (a)  if  a  MCA  method  deals 
explicitly  with  uncertainty,  (b)  if  more  than  one  decision-maker 
is  involved  and  (c)  the  moment  when  modeling  of  C&I  preferences 
through  stakeholder's  involvement  takes  place.  In  regard  to  the 
last  aspect,  Mendoza  and  Martins  [7]  reviewed  academic  papers 
using  three  different  categories:  (a)  ‘a  priori’,  (b)  interactive  and 
(c)  posterior  articulations.  ‘A  priori’  means  that  the  stakeholder 
involvement  starts  with  the  modeling  of  preferences.  On  the  other 
hand,  ‘interactive  articulation’  of  preferences  results  in  the  most 
preferred  solution  or  alternative  through  stakeholders  answering 
specific  questions.  For  ‘posterior  articulation’  of  preferences  ‘non- 
dominated’  solutions  are  generated  prior  to  the  evaluations  made 
by  the  stakeholders.  When  elucidation  methods  are  carried  out  in 
group  decision  making  situations  the  individual  stakeholder  pre¬ 
ferences  can  be  synthesized  in  various  ways,  whereas  Mendoza 
and  Martins  [7]  mention  that  aggregating  has  been  established  as 
a  main  aspect  in  methodological  adaptations  of  MCA  to  group 
decision  making.  Furthermore,  they  point  out  that  many  aspects  in 
the  field  of  natural  resources  management  cannot  be  estimated  or 
predicted  sufficiently,  which  contributes  to  uncertainty  in  decision 
making.  However,  if  uncertainty  is  not  addressed  sufficiently, 
suboptimal  or  even  inappropriate  alternatives  are  potentially 
selected  and  implemented  ultimately  resulting  in  unsustainable 
outcomes.  Other  selection  criteria  mentioned  [7]  (e.g.  ‘country  of 
application’,  ‘spatial  scale’,  ‘nature  and  context  of  problems’  or 
‘type  of  data  used’)  provide  more  descriptive  information  and, 
therefore,  are  less  relevant  for  the  selection  of  MCA  methods. 


The  mentioned  criterion  ‘type  and  selection  of  stakeholders’  can 
also  be  regarded  as  being  irrelevant  for  MCA  method  selection, 
this  process  usually  takes  place  as  a  separate  step  earlier  in  a 
decision  making  process  (e.g.  [2,30,31]). 

Buchholz  et  al.  [6]  analyzed  how  applicable  MCA  methods  for 
bioenergy  systems  assessments  are  by  scrutinizing  their  structural 
approach  using  nine  selection  criteria.  The  first  selection  criterion 
covers  ‘stakeholder  inclusion  in  decision  making’  and  takes 
account  of  the  fact  that  renewable  energy  developments,  such  as 
bioenergy  developments,  typically  involve  a  relatively  high  num¬ 
ber  of  stakeholders.  In  this  connection,  it  is  pointed  out  that  many 
of  past  projects  were  unsuccessful  when  stakeholders  were  not 
engaged  appropriately.  The  second  selection  criterion  covers  the 
‘application  of  qualitative  data’.  Technical  aspects  are  taken  into 
account  by  three  criteria.  ‘Measures  to  deal  with  uncertainty’  is  the 
first  among  those  and  is  justified  by  the  significance  and  varied 
forms  of  uncertainty,  which  can  be  dealt  with  by  MCA  methods  at 
the  different  stages  of  a  decision  making  process.  Another  techni¬ 
cal  aspect  is  the  ‘inclusion  of  criteria  hierarchies’.  Hierarchies  assist 
in  reducing  complexities  and  allow  facilitated  communication  of 
decision  making  results  and  strategies  to  stakeholders  without 
technical  knowledge.  The  third  technical  criterion  ‘use  of  thresh¬ 
olds’  is  about  implications  of  applied  numerically  precise  thresh¬ 
olds,  which  are  helpful  in  dealing  with  data  uncertainty.  Four 
criteria  cover  the  applicability  of  MCA  methods.  ‘Ease  of  computa¬ 
tion’  addresses  the  technical  user-friendliness  and  flexibility  of 
MCA  methods.  In  this  respect,  the  criterion  also  takes  into  account 
the  differences  among  MCA  methods  regarding  problem  structur¬ 
ing,  parameter  setting  and  sensitivity  analysis.  Related  to  this 
criterion  is  ‘dynamic  re-evaluation’,  which  is  about  a  MCA  method's 
ability  to  allow  re-evaluating  decisions  based  on  new  structures, 
insights  or  data.  This  ability  supports  flexible  and  adaptive  learn¬ 
ing  cycles  as  proposed  by  Gamboa  and  Munda  [32],  An  important 
aspect  when  involving  numerous  stakeholders  is  the  application  of 
simple  math  and  intuitive  methods  to  improve  the  acceptance  of 
MCA  methods  results.  The  criterion  ‘transparency’  covers  this 
aspect.  Related  to  this  criterion  is  the  last  criterion  ‘communica¬ 
tion  of  decision  process  and  results’.  Decent  abilities  of  MCA 
methods  in  respect  to  this  aspect  can  be  critical  when  numerous 
stakeholders  are  involved.  Apart  from  aspects  related  to  stake¬ 
holder  involvement  and  easy  handling,  considering  these  aspects 
covering  acceptance  and  understanding  are  also  recommended  by 
Huang  et  al.  [13]  under  certain  circumstances. 

Ozcan  et  al.  [16]  describe  a  comparative  analysis  of  MCA 
(specifically  MCDM)  methodologies  and  implement  a  warehouse 
location  selection  problem.  Although,  not  specifically  focused  on 
sustainability  or  energy  decision  making,  their  analysis  and 
comparison  of  MCA  methods  including  AHP,  TOPSIS,  ELECTRE 
I— III  provide  useful  inputs  for  the  process  of  selecting  an  appro¬ 
priate  MCA  method.  The  several  methods  reviewed  in  the  paper 
are  differentiated  by  seven  characteristics  or  criteria:  (a)  core 
process,  (b)  necessity  to  quantify  the  relative  importance  of  C&I, 
(c)  determining  of  C&I  weights,  (d)  number  and  type  of  outranking 
relations,  (e)  consistency  check,  (f)  problem  structure  and  (g)  final 
results.  The  first  characteristic  provides  rather  descriptive  infor¬ 
mation  and  can  be  regarded  as  being  less  suitable  for  MCA  method 
selection.  The  criteria  about  quantifying  the  relative  importance  of 
C&I  and  determining  C&l  weights  are  both  focusing  on  the  ‘MCA 
cycle’  step  ‘C&I  weighting  or  ranking’.  The  first  criterion  does  not 
serve  to  discriminate  sufficiently  between  methods  to  support  the 
selection  process,  because  almost  all  MCA  methods  require  the 
relative  importance  of  C&I  to  be  quantified.  On  the  other  hand, 
the  second  criterion  can  be  considered  as  being  important, 
because  C&I  weighting  is  an  essential  part  of  MCA  methods  and 
differences  between  methods  in  this  regard  are  significant.  Not 
every  MCA  method  has  a  specific  weighting  method  incorporated 
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and  for  some  C&I  weights  are  provided  without  any  calculations. 
However,  for  the  latter  methods  commonly  linear  and  vector 
normalizations  are  used  [33].  The  described  criterion  ‘number 
and  type  of  outranking  relations’  can  be  regarded  as  being  too 
specific  and  focused  on  outranking  methods  for  selecting  MCA 
methods.  On  the  other  hand,  it  can  be  regarded  as  being  important 
to  measure  and  control  the  consistency  of  stakeholders'  prefer¬ 
ences.  The  criterion  ‘consistency  check’  shows,  if  MCA  methods 
allow  controlling  consistency.  The  criterion  ‘problem  structure’ 
covers  numbers  of  C&l  and  alternatives,  which  have  to  be 
compared,  as  well  as  a  method's  ability  to  handle  quantitative 
and/or  qualitative  data.  This  aspect  is  only  relevant  for  decision 
making  problems  with  large  numbers  of  C&I  and  alternatives,  and 
as  mentioned  previously,  only  if  qualitative  data  is  used.  The 
criterion  ‘final  results’  covers  the  shape  in  which  acquired  MCA 
results  are  available.  This  criterion  can  be  regarded  as  a  non- 
essential  feature  for  selecting  suitable  MCA  methods. 

3.  Application  of  the  methodology  for  MCA  method  selection 

This  section  illustrates  how  the  methodology  for  MCA  method 
selection  was  applied.  First,  the  case  study  is  described.  Second, 
the  most  appropriate  set  of  MCA  method  selection  criteria  was 
developed  based  on  the  reviewed  literature  (Section  2.2).  Then, 
some  MCA  methods  were  pre-selected  and  compared  against 
these  criteria  to  choose  the  most  appropriate  MCA  method  for 
the  case  study. 

3.1.  Case  study  description 

For  the  application  of  the  methodology  to  select  a  MCA 
method,  basically  a  generic  decision  making  situation  was 
assumed,  in  which  renewable  energy  developments  are  planned. 
In  terms  of  the  MCA  method  selection  itself,  it  was  further 
assumed  that  MCA  methods  cover  the  following  steps  of  the 
MCA  cycle:  (i)  C&I  weighting  or  ranking,  (ii)  assessment  and 
ranking  of  alternatives  and  (iii)  sensitivity  analysis.  Also,  it  was 
assumed  that  stakeholders  are  involved  in  a  single  event  for  C&I 
weights  determination.  In  terms  of  C&I  performance  assessments, 
it  was  assumed  that  the  performances  are  assessed  objectively  and 
that  stakeholders  agree  exclusively  on  employing  quantitative  C&l. 
Choosing  this  generic  decision  making  situation  was  intended  to 
demonstrate  the  broad  application  potential  of  the  presented 
approach  to  select  MCA  methods  in  the  renewable  energy  sector. 

3.2.  Choice  of  criteria  for  MCA  method  selection 

Based  on  the  literature  review  (Section  2.2),  a  set  of  most 
appropriate  MCA  selection  criteria  for  the  case  study  was  devel¬ 
oped.  Table  3  illustrates  these  criteria,  which  are  also  briefly 
described  for  clarification. 

The  illustrated  selection  criteria  were  chosen  by  considering 
their  relevance  for  the  case  study,  but  also  by  bearing  in  mind  how 

Table  3 

Criteria  for  MCA  method  selection  for  the  case  study. 


independent  they  are  from  each  other.  For  instance,  criteria  related 
to  ‘transparency’,  ‘inclusion  of  C&I  hierarchies’  and  ‘communica¬ 
tion  of  decision  process  and  results’  can  be  employed  as  separate 
criteria.  However,  when  looking  at  the  descriptions  of  criteria  [6], 
relatively  strong  interrelationships  can  be  identified.  Good  com¬ 
munication  leading  to  easy  comprehension  is  supported  by  trans¬ 
parency,  but  on  the  other  hand  good  communication  assists  in 
increasing  acceptance  and  transparency.  C&I  hierarchies  help  to 
reduce  complexity  and  as  a  consequence  facilitate  communication 
or  processes  and  results.  This  again  improves  stakeholder  com¬ 
prehension.  Therefore,  for  the  case  study  these  three  criteria  were 
merged  into  a  single  criterion  called  ‘transparency  and  commu¬ 
nication’.  In  general,  the  selection  criteria  were  kept  as  independent 
as  possible  for  the  case  study.  However,  some  interrelationships 
exist.  For  instance  the  criterion  ‘multi-stakeholder  inclusion’  is 
related  to  the  criteria  ‘user  friendliness  and  flexibility’,  as  well  as 
‘transparency  and  communication’  to  a  certain  extent.  Also,  it  has 
to  be  highlighted  that  a  selection  criterion  comparing,  whether 
MCA  methods  are  able  to  process  qualitative  C&I  data  was  not 
required,  because  as  mentioned  (Section  3.1)  only  quantitative  C&I 
were  considered.  Also,  employing  exclusively  quantitative  C&I, 
which  can  be  assessed  objectively,  affected  the  scoring  of  the 
MCA  methods  against  the  selection  criterion  ‘measures  to  deal 
with  uncertainty’,  as  uncertainty  related  to  determining  C&I 
weights  could  be  focused  on.  This  demonstrates  that  the  processes 
of  choosing  appropriate  selection  criteria  and  the  selection  of  MCA 
methods  itself  depend  heavily  on  a  project’s  or  case  study's 
specific  circumstances.  In  this  connection,  particularly,  the  way 
multiple  stakeholders  were  involved  plays  a  major  role.  For 
instance,  if  stakeholders  want  to  be  involved  in  a  single  event 
for  C&I  weights  determination  -  as  it  was  assumed  in  case  study's 
decision  making  situation  (Section  3.1 )  -  C&I  weighting  has  to  be 
undertaken  ‘a  priori’.  Therefore,  ‘a  priori’  C&I  weighting  had  to  be 
considered  as  a  prerequisite  in  selecting  MCA  methods  in  the  case 
study.  Similarly,  in  such  a  decision  making  situation  in  which 
stakeholders  agree  exclusively  on  quantitative  C&I  with  relatively 
certain  performance  values,  capabilities  of  MCA  methods  to  deal 
with  qualitative  C&I  and  uncertainty  are  not  required  or  only 
relevant  to  a  certain  degree,  respectively. 

3.3.  Pre-selection  of  MCA  methods 

In  this  section  it  is  described  how  three  MCA  methods  were 
pre-selected  in  order  to  test  the  methodology  in  the  case  study. 
This  pre-selection  was  mainly  based  on  the  popularity  of  MCA 
methods. 

Huang  et  al.  [13]  identified  and  classified  over  300  papers  on 
environmental  applications  of  MCA  (specifically  MCDA),  which  were 
published  between  2000  and  2009.  The  papers  were  classified  by  their 
environmental  application  area,  decision  or  intervention  type  and 
three  main  MCA  approaches  were  identified  and  employed  for  their 
analysis:  ‘Outranking’,  ‘AHP/ANP  (Analytic  Network  Process)’  and 
‘MAUT/MAVT’.  When  looking  at  papers  about  MCA  methods  in  the 
application  area  ‘Energy’  published  between  2000  and  2009,  AHP/ANP 


Selection  criteria 


Description: 


Measures  to  deal  with  uncertainty 
User-friendliness  and  flexibility 

Transparency  and  communication 


Multi-stakeholder  inclusion 


To  what  degree  does  a  MCA  method  address  uncertainty  in  its  different  stages? 

How  easy  is  the  computation  of  MCA  processes?  Is  the  MCA  method  user  friendly,  flexible  and  adaptive  and  does  it  allow 
re-evaluations? 

How  simple  and  intuitive  is  a  MCA  method  to  increase  transparency  and  stakeholder  acceptance?  Are  hierarchies 
incorporated  to  help  reducing  complexity  and  to  support  communication  of  processes  and  results  in  order  to  improve 
stakeholder  comprehension? 

How  easy  is  it  to  involve  multiple  stakeholders  in  the  MCA  process? 
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approaches  were  predominantly  employed,  followed  by  PROMETHEE, 
ELECTRE  and  MAUT/MAVT  methods  accounting  for  42,  12,  9  and  9% 
respectively.  Moreover,  a  trend  towards  AHP/ANP  methods,  indicated 
by  the  growing  number  of  their  applications  over  that  decade,  was 
identified.  AHP/ANP  methods  were  also  identified  as  the  most  popular 
approaches  (48%)  when  looking  at  published  papers  about  applica¬ 
tions  of  MCA  methods  with  stakeholder  participation. 

For  the  case  study,  most  popular  MCA  methods  according  to 
Huang  et  al.  [13]  were  compared  against  the  selection  criteria 
(Table  3).  These  pre-selected  MCA  methods  were:  AHP  [34],  DELTA 
[35]  and  PROMETHEE  [36]  representing  MAVT,  MAUT  and  out¬ 
ranking  methods,  respectively  [37],  In  order  to  compare  only  one 
method  representing  a  main  MCA  approach,  the  popular  ELECTRE 
method  was  excluded  from  the  comparison  due  to  its  lower 
popularity  compared  to  PROMETHEE,  which  is  also  an  outranking 
method.  Also,  AHP  was  focused  on  in  this  comparison  rather  than 
ANP,  because  AHP  is  significantly  more  applied  in  the  environ¬ 
mental  management  and  agriculture  application  area  [38], 

Notable  is  that  MCA  combined  fuzzy  methodologies,  which 
involve  a  series  of  mathematical  steps  and  are  basically  fuzzy 
extensions  to  conventional  MCA  methods  (e.g.  fuzzy  outranking 
methods  and  fuzzy  utility  theory),  have  also  been  applied  in 
energy  decision  making  situations  various  times  (e.g.  [25,39,40]). 
However,  due  to  several  reasons  they  are  unpopular  within  the 
MCA  community  and  have  predominately  been  discussed  in 
academic  literature  and  conducted  experimentally  [2],  For 
instance,  particularly  for  AHP  and  ANP  the  accuracy  of  the  fuzzy 
approach  is  criticized  [41  ].  Therefore,  the  practical  value  of  MCA 
combined  fuzzy  methodologies  can  be  substantially  questioned 
and  it  is  even  assumed  that  their  future  application  will  be  flawed 
in  regard  to  better  decision  making  [38],  Consequently,  these 
approaches  were  excluded  from  the  process  of  selecting  MCA 
methods  in  the  case  study. 

3.4.  Scoring  of  pre-selected  MCA  methods 

Each  of  the  three  pre-selected  MCA  methods  ‘AHP’,  ‘DELTA’  and 
‘PROMETHEE’  was  compared  against  the  four  selection  criteria 
described  previously  (Table  3).  In  this  process,  the  MCA  methods 
were  assessed  using  a  three-point  scale  with  Tow’,  ‘medium’  and 
‘high’  scores.  Additionally,  specific  freely  downloadable  or  com¬ 
mercially  available  software  packages  were  considered  to  compare 
them  against  some  of  the  selection  criteria: 

•  ‘Super  Decisions’  (AHP) 

•  ‘DecidelT’  (DELTA  method) 

•  'Decision  Lab’  (PROMETHEE  11) 

The  software  packages  were  selected  due  to  their  common 
applications  in  related  fields  such  as  renewable  energy,  natural 
resource  management  or  participatory  assessments.  Based  on  a 
literature  review  and  by  considering  the  case  study’s  circum¬ 
stances  (Section  3.1),  the  software  packages'  scoring  against  some 
of  the  selection  criteria  contributed  to  identifying  the  most 
suitable  MCA  method  and  its  software  package  for  the  case  study. 
The  following  two  examples  illustrate  how  the  scoring  against  two 
selection  criteria  was  undertaken. 


3.4.1.  User  friendliness  and  flexibility 

‘User-friendliness  and  flexibility’  was  one  of  the  four  criteria 
against  which  the  AHP,  DELTA  and  the  PROMETHEE  II  methods 
were  compared.  The  scores  for  this  criterion  reflect  several  aspects 
including  ease  of  computation  and  technical  user-friendliness  of 
the  reviewed  methods  and  their  software  packages,  as  well  as 


their  flexibility  and  adaptability  in  terms  of  allowing  re-evaluation 
of  decisions  with  new  data  inputs  or  structures. 

‘Super  Decisions’  scored  ‘medium’  against  this  criterion.  It 
allows  easy  computation  with  its  straightforward  structure  and 
its  simple  framework  to  build  and  assess  scenarios  and  alterna¬ 
tives.  Particularly,  data  input  of  the  pairwise  comparison  data  is 
perceived  as  simple  and  convenient  by  users  [2],  Furthermore, 
data  input  requirements  are  minimal  and  free  downloadable 
software  package  versions  of  ‘Super  Decisions’  are  available. 
Looking  at  these  aspects  in  isolation  the  user  friendliness  of  ‘Super 
Decisions’  is  high.  However,  it  is  relatively  difficult  to  apply 
sensitivity  analyses  and  dynamic  re-evaluations.  Performing  sen¬ 
sitivity  analyses  is  difficult,  because  the  software  package's  cap¬ 
ability  for  analysis  is  restricted  to  one  criterion  at  a  time.  In  terms 
of  dynamic  re-evaluations,  direct  comparisons  between  existing 
assessments  and  assessments  with  new  data  input  are  not 
possible.  Comparing  those  assessments  is  only  feasible  indirectly 
through  re-structuring  data  and  running  separate  assessments 
[6,15].  Also,  the  ‘rank  reversal’  phenomenon  [42]  has  to  be 
considered  when  decisions  want  to  be  re-evaluated.  However,  this 
phenomenon  has  only  to  be  taken  account  of,  in  cases  where  C&I 
performance  values  are  included  in  the  assessments. 

In  terms  of  ease  of  computation  and  technical  user-friendliness, 
the  capabilities  of  the  commercially  available  DecidelT  software 
package  can  be  considered  as  decent  with  its  advanced  user 
interface  and  intuitive  logic  to  structure  and  define  C&I,  as  well 
as  conducting  sensitivity  analysis.  Furthermore,  it  requires  low 
minimum  data  input.  However,  if  minimization  is  the  goal  of  the 
decision  making  situation,  this  software  package  of  the  DELTA 
method  requires  manual  inversion  of  data.  For  example,  the  goal 
air  pollution  reduction  has  to  be  converted  to  air  quality  improve¬ 
ment.  This  means  an  additional  step,  which  reduces  DecidelT’s 
user  friendliness.  As  for  ‘Super  Decisions’,  re-evaluating  decisions 
is  difficult  as  new  data  input  cannot  be  integrated,  which  disallows 
direct  comparisons  between  old  and  new  assessments  [6],  There¬ 
fore,  the  case  study's  score  for  ‘user  friendliness  and  flexibility’  was 
considered  to  be  ‘medium’. 

The  available  software  package  ‘Decision  Lab’  scored  ‘high’ 
against  ‘user  friendliness  and  flexibility’.  It  allows  relatively  easy 
computation  in  all  its  application  stages.  Technical  user  friendli¬ 
ness  is  supported  by  an  easy-to-use  interface  to  structure  the 
decision  making  problem  and  to  input  data.  This  software  package 
of  PROMETHEE  II  also  offers  a  user-friendly  sensitivity  analysis 
with  the  possibility  to  have  numerous  perspectives  on  the  pro¬ 
blem.  This  includes  the  use  of  ‘walking  weights’  and  side-to-side 
assessments  of  scenarios  and  alternatives  [6,15],  In  terms  of 
dynamic  re-evaluation,  ‘Decision  Lab’  is  equally  well  capable, 
which  allows  engaging  in  flexible  and  adaptive  learning  cycles.  It 
is  the  only  reviewed  software  package,  which  offers  direct  com¬ 
parisons  between  existing  and  new  assessments  of  scenarios  and 
alternatives  through  its  scenarios  and  alternatives  analysis,  which 
also  allows  structures  to  be  changed  for  these  comparisons  [6[. 


3.4.2.  Multi-stakeholder  inclusion 

Another  criterion  against  which  the  three  MCA  methods  AHP, 
DELTA  and  PROMETHEE  II  were  compared  against  was  ‘multi¬ 
stakeholder  inclusion’.  This  criterion  is  about  an  approach's  cap¬ 
ability  to  involve  multiple  stakeholders  in  the  MCA  process  easily. 
Naturally,  the  two  criteria  ‘user  friendliness  and  flexibility',  as  well 
as  ‘transparency  and  communication’  are  connected  to  this  criterion 
to  a  certain  extent.  Thus,  the  scores  of  the  methods  against  those 
criteria  also  contain  and  cover  elements  of  the  approaches' 
abilities  to  include  multiple  stakeholders.  In  line  with  the  scenario 
description  (Section  3.1),  the  three  MCA  methods  AHP,  DELTA  and 
PROMETHEE  II  and  their  software  packages  were  evaluated 
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regarding  their  coverage  of  some  or  all  of  the  mentioned  steps  of 
the  MCA  cycle.  In  other  words  a  holistic  perspective  looking  at  all 
those  stages  was  necessary  to  determine  an  approach's  capability 
for  multi-stakeholder  involvement.  Also  in  line  with  the  scenario 
description  (Section  3.1),  it  has  to  be  taken  into  account  that  it  was 
assumed  that  stakeholders  express  their  opinions  towards  C&l 
priorities  ‘a  priori’.  All  three  reviewed  MCA  methods  allow  the 
incorporation  of  C&I  weighting  during  that  stage  of  the  decision 
making  process. 

The  AHP  method  scored  ‘high’  against  the  selection  criterion 
when  keeping  the  case  study  in  mind.  One  of  the  core  elements  of 
AHP  is  the  conversion  of  subjective  assessments  of  relative 
importance  to  overall  weights  [2,6].  The  method  is  the  only  of 
the  three  reviewed  MCA  methods,  which  works  with  a  specific 
procedure  to  determine  C&l  weights  -  pairwise  comparisons 
between  C&I  -  and  through  that  provides  ready-to-use  C&I 
weights  or  rankings.  Pairwise  comparisons  significantly  reduce 
the  likelihood  of  stakeholders  making  mistakes  and  rough  estima¬ 
tions  by  deriving  ratio  scale  priorities,  while  eliciting  utility 
functions  is  not  required  [17],  This  allows  stakeholders  to  be  easily 
involved.  The  method  also  allows  group  decision  making  proce¬ 
dures  in  assessing  and  ranking  alternatives,  as  well  as  in  the 
sensitivity  analysis  stage  [6,7], 

The  DELTA  method  scored  ‘medium’  against  the  criterion 
'multi-stakeholder  inclusion’.  This  score  is  justified,  because  the 
method  requires  external  aggregation  of  C&I  weights  through 
pairwise  comparison.  This  makes  stakeholder  involvement  more 
difficult.  However,  the  method  allows  stakeholder  involvement  in 
scenarios  and  alternatives  assessment  and  sensitivity  analysis, 
when  uncertainty  of  C&I  weights  is  taken  account  of  [6], 

The  PROMOTHEE  II  approach  also  scored  ‘medium’  against  the 
criterion  ‘multi-stakeholder  inclusion’.  Like  the  DELTA  method  it 
relies  on  externally  derived  C&I  weights  or  rankings.  However, 
stakeholders  can  be  involved  in  the  assessment  of  scenarios  and 
alternatives,  as  well  as  in  the  sensitivity  analysis.  Furthermore,  it 
allows  analyzing  group  decisions  and  comparing  individual  opi¬ 
nions  with  each  other  by  assigning  scenarios  to  each  individual 
stakeholder’s  opinion  [6], 


4.  Results  and  discussion 

The  scores  of  the  three  different  MCA  methods  compared 
against  the  selection  criteria  are  summarized  below  (Table  4): 

In  the  case  study,  the  AHP  method  scored  best  with  three  ‘high’ 
scores  and  one  ‘medium’  score,  followed  by  the  DELTA  method 
with  two  ‘high’  and  two  ‘medium’  scores.  PROMETHEE  II  only 
scored  ‘high’  once,  ‘medium’  twice  and  Tow’  against  one  criterion. 
Interestingly,  this  MCA  method  scored  best  against  the  criterion 
‘user  friendliness  and  flexibility’,  against  which  the  other  methods 
only  achieved  a  ‘medium’  score.  Conversely,  PROMETHEE  II  scored 
worst  against  ‘measures  to  deal  with  uncertainty’,  whereby  the 
results  of  the  other  two  methods  are  particularly  well  against  this 
criterion.  Further,  it  can  be  observed  that  the  scores  for  the  AHP 
and  Delta  methods  against  the  selection  criteria  only  differ  in 


Table  4 

Scoring  results  of  reviewed  MCA  methods  for  the  case  study. 


Selection  criterion 

MCA  method 

AHP 

DELTA 

PROMETHEE  II 

Measures  to  deal  with  uncertainty 

High 

High 

Low 

User  friendliness  and  flexibility 

Medium 

Medium 

High 

Transparency  and  communication 

High 

High 

Medium 

Multi-stakeholder  inclusion 

High 

Medium 

Medium 

regard  to  ‘multi-stakeholder  inclusion’,  which  is  not  surprising 
when  bearing  in  mind  how  closely  MAVT  and  MAUT  are  related 
[1],  The  difference  in  scores  is  mainly  due  to  the  fact  that  AHP  is 
the  only  of  the  three  methods,  which  works  with  a  specific 
participatory  procedure  to  determine  C&I  weights  —  pairwise 
comparisons  —  instead  of  relying  on  external  aggregation  of  C&I 
weights. 

When  reflecting  and  concluding  on  the  scoring  process,  it  has 
to  be  addressed  that  several  factors  and  assumptions  influenced 
the  results  significantly.  First,  it  has  to  be  considered  against  what 
the  MCA  methods  were  compared,  i.e.  the  selection  criteria. 
Choosing  them  was  to  a  certain  extent  subjective  and  their  choice 
influenced  the  results  of  the  MCA  method  selection  substantially. 
In  other  words,  using  different  criteria  against  which  the  methods 
were  compared,  could  have  led  to  diverging  results.  The  same 
applies  to  what  was  scrutinized  against  the  criteria,  i.e.  the  three 
MCA  methods.  Although,  their  pre-selection  was  justified  by  their 
popularity  and  common  use  in  the  energy  sector,  the  focus  on 
specific  software  packages  representing  those  methods  when 
compared  against  some  of  the  selection  criteria,  was  rather 
subjective.  However,  in  either  instance  the  subjectivity  stemmed 
from  the  case  study’s  decision  making  situation,  which  had  to  be 
considered  (Section  3.1).  Huang  et  al.  [13]  confirm  this  by  high¬ 
lighting  the  importance  of  considering  the  complexity  of  the  MCA 
method  selection  in  respect  to  influencing  scientific,  social  and 
technical  factors,  but  also  in  regard  to  an  understanding  of  the 
process  requirements,  as  well  as  the  degree  of  available  knowledge 
about  the  specific  decision  making  problem.  For  the  case  study  the 
most  influencing  factors  in  this  respect  were  the  fact  that  the  focus 
was  on  C&l  weights  excluding  C&I  performance  values,  as  well  as 
quantifiable  C&L  In  case  the  stakeholders  would  have  chosen 
predominantly  qualitative  C&I,  another  MCA  method  selection 
process  resulting  in  different  outcomes  would  have  been  required. 
Features  of  MCA  methods  in  regard  to  qualitative  C&I,  as  well  as 
the  incorporation  of  C&I  performance  values  in  the  decision 
making  process  was  therefore  not  important  to  the  different  stages 
of  the  MCA  method  selection  process  in  the  case  study.  This  again 
demonstrates  that  the  process  of  choosing  appropriate  MCA 
methods  heavily  depends  on  a  project's  or  case  study's  specific 
circumstances,  which  is  also  confirmed  by  Guitouni  and  Martel 
[15]  who  state  a  lack  of  a  single  ideal  and  appropriate  MCA  method 
for  all  decision  making  situations.  Consequently,  it  could  be  argued 
that  the  complex  process  of  selecting  the  most  suitable  MCA 
method  could  paradoxically  call  for  applying  a  MCA  method  to 
help  choosing.  This  however  would  result  in  a  ‘chicken-or-egg 
situation',  because  to  employ  a  MCA  method  to  help  choosing  a 
MCA  method  would  imply  selecting  a  MCA  method  in  the  first 
place,  which  would  still  leave  one  wondering  how  to  select  that 
method.  Further,  it  can  be  observed  that  the  selection  criteria 
deemed  to  be  relevant  (Section  2.2)  were  predominantly  criteria 
with  emphasis  on  acceptance,  understanding  and  interaction  with 
the  MCA  method.  Compared  against  them  AHP  is  a  strong  MCA 
method.  Buchholz  et  al.  [6]  and  Chatzimouratidis  and  Pilavachi 
[17]  explain  that  the  method’s  general  popularity  and  wide  use  is 
mainly  due  to  its  logical  approach  and  simplicity  facilitating  these 
aspects,  whereas  Huang  et  al.  [13]  regard  available  software 
packages,  expertise  and  particularly  active  user  groups  as  possible 
reasons  for  this  popularity.  In  this  connection  it  is  notable  that 
AHP  lacks  qualities  in  regard  to  addressing  dependencies  and 
interrelations  between  C&L  Conversely,  its  extension  ANP  has 
distinctive  qualities  in  this  regard,  which  can  lead  to  substantial 
improvements  in  decision  making.  Due  to  this  reason  it  is 
expected  that  ANP  will  gain  more  popularity  compared  to  AHP 
in  the  future  [38],  However,  due  to  ANP's  higher  complexity,  ‘user 
friendliness  and  flexibility’,  ‘transparency  and  communication'  as 
well  as  ‘multi-stakeholder  inclusion’  can  be  regarded  as  being 
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reduced  compared  to  AHP.  As  mentioned  (Section  2.2),  Huang 
et  al.  [13]  advise  MCA  users  to  select  methods  based  on  aspects 
related  to  stakeholder  involvement,  acceptance,  easy  handling  and 
understanding  when  MCA  methods  results  can  be  expected  to  be 
robust.  Such  a  robustness  of  MCA  methods  is  generally  the  case. 
When  applying  different  MCA  methods  to  the  same  decision 
making  problem,  it  was  observed  that  although  the  ranking  of 
alternatives  may  change,  generally  the  same  alternatives  come  out 
at  the  top  regardless  of  the  MCA  method  employed  [13].  This  is 
due  to  the  mathematical  phenomenon  that  the  best  ranking 
alternative(s)  is/are  often  so  much  better  than  the  remaining 
alternatives  so  that  the  differences  between  MCA  methods  are 
too  minor  to  significantly  change  the  outcome.  The  reason  for  this 
is  that  those  differences  are  most  probably  affecting  all  alterna¬ 
tives  equally  within  C&l  assessment  levels  unless  all  MCA  methods 
would  be  biased  towards  a  single  alternative,  which  is  highly 
unlikely  [43,44].  The  phenomenon  was  particularly  observed  for 
decision  making  situations  with  a  small  number  of  alternatives 
[13].  Therefore,  for  these  cases  distinctive  qualities  of  MCA 
methods  would  be  less  significant  than  for  instance  selecting  most 
appropriate  C&I  or  involving  appropriate  stakeholders.  Neverthe¬ 
less,  regardless  the  number  of  alternatives,  selecting  a  suitable 
MCA  method  in  the  described  systematic  manner  is  an  effective 
and  valid  approach  for  decision  making  situations,  because  it 
illustrates  and  justifies  why  a  particular  MCA  method  makes  more 
sense  than  other  compared  methods  when  specific  aspects  are 
relevant  and  focused  on.  Furthermore,  when  applying  a  MCA 
method,  which  was  selected  by  employing  the  presented  metho¬ 
dology,  the  identified  limitations  and  disadvantages  of  the  method 
with  respect  to  the  selection  criteria  can  be  considered  more 
easily.  This  can  facilitate  processes  in  similar  decision  making 
situations,  particularly  in  the  renewable  energy  sector.  In  this 
regard,  the  presented  methodology  is  advantageous  compared  to 
other  approaches  to  select  MCA  methods  (e.g.  [6]),  due  to  its 
generic  nature.  Although,  similar  MCA  selection  criteria  are  pre¬ 
sented  in  related  works  [6,7,15,16],  a  methodology  to  system¬ 
atically  select  MCA  methods  by  developing  an  appropriate  set  of 
selection  criteria,  pre-selecting  MCA  methods  and  scoring  the  pre¬ 
selected  methods  against  the  developed  set  of  selection  criteria 
has,  to  our  knowledge,  not  been  published  yet. 


5.  Conclusions 

This  paper  provides  a  generic  approach  to  select  a  suitable  MCA 
method  covering  the  stages  of  the  ‘MCA  cycle’  for  decision  making 
in  the  renewable  energy  sector.  Selecting  a  suitable  MCA  method 
for  an  individual  project  or  case  study  is  challenging  due  to  the 
complexity  of  participatory  sustainable  decision  making  in  this 
sector,  as  well  as  due  to  the  complexity  and  large  quantity  of 
existing  MCA  methods. 

The  provided  methodology  for  selecting  MCA  methods  consists 
of  both,  generic  and  case-study  specific  parts.  First,  it  is  proposed 
to  identify  and  review  MCA  methods  and  criteria  to  select  MCA 
methods  from  literature.  Second,  case  study  specific  steps  are 
proposed  to  be  applied  including:  development  of  a  set  of  appro¬ 
priate  criteria  for  MCA  method  selection,  pre-selection  of  MCA 
methods  and  scoring  of  the  pre-selected  MCA  methods  against  the 
developed  selection  criteria. 

In  general,  the  process  of  selecting  appropriate  MCA  methods 
depends  heavily  on  a  project’s  or  case  study's  specific  circum¬ 
stances.  However,  the  case  study's  decision  making  situation 
presented  in  this  paper  was  deliberately  developed  in  a  rather 
generic  way  allowing  a  wide  range  of  application  potential  for 
decision  making  situations  within  the  renewable  energy  sector. 
The  criteria  for  MCA  method  selection  deemed  to  be  relevant  were 


predominantly  criteria  focusing  on  acceptance,  understanding  and 
interaction  with  the  MCA  method.  AHP  scored  best  when  pre¬ 
selected  MCA  methods  were  compared  against  the  selection 
criteria. 

Although,  in  certain  circumstances  the  results  of  MCA  methods 
do  not  differ  significantly,  selecting  suitable  MCA  methods  in  the 
proposed  systematic  manner  is  an  effective  and  valid  approach, 
because  it  helps  focusing  on  specific  aspects,  which  are  considered 
important  and  relevant  for  decision  making  processes.  Therefore, 
for  similar  research  or  decision  making  situations  the  proposed 
methodology  for  MCA  selection  can  be  employed,  although 
different  criteria  and/or  MCA  methods  may  be  chosen  for  the 
selection  process.  Furthermore,  the  identified  and  reviewed  MCA 
methods  and  selection  criteria  provide  a  broad  application  poten¬ 
tial  for  MCA  method  selection  in  the  energy  and  renewable  energy 
sectors. 
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